Fault Tolerance of Feedforward Neural Nets for Classification Tasks

A method is proposed to estimate the fault tolerance of feedforward Artificial Neural Nets (ANNs) and synthesize robust nets. Fault models are presented and a procedure is developed to build fault tolerant ANNs by replicating the hidden units. Based on this procedure, metrics are devised t o quantify the fault tolerance aa a function of redundancy. A significant amount of redundancy is shown to be necessary to achieve complete fault tolerance even if only single faults are considered. Furthermore, lower bounds on the required redundancy are analytically derived for some canonical problems. Our results indicate that ANNs have good partial fault tolerance and degrade gracefully. In particular, just one extra replication is seen to considerably improve the fault tolerance. Neural computing is rapidly evolving as a viable solution to several problems. For use in applications requiring high reliability, ANNs have to be fault tolerant. The ANN should possess a high degree of fault tolerance to begin with and its performance should degrade gracefully with increasing number of faults. Several expositions have addressed various aspects of fault tolerance of ANNs [1]-[7]. Investigations by Carter et. al. [l, 21 demonstrate the need to quantitatively evaluate the fault tolerance of ANNs. Simulation results on the XOR problem are reported in [3], but these are quite specific to this one example and the underlying fault model. Fault tolerance of Hopfield type ANNs for optimization problems was investigated in [4]. It, however, does not address fault tolerance in trainable ANNs. Belfore et. al. have developed an analytical technique for estimating the performance of ANNs in presence of faults [5, 61 They construct a Markov model for the ANN by drawing analogy with magnetic spin systems using statistical mechanics. Neti et. al. recently reported numerical results on the synthesis of fault tolerant nets [7]. They add constraints to ensure fault tolerance and look for a solution to the constrained optimization problem. This approach, however, is very computation intensive and more experiments are needed to draw any general conclusions about fault tolerance. Most researchers concentrate on performance estimation in the presence of faults for a given net. To the best of our knowledge, there has been no attempt to relate the fault tolerance to the redundancy necessary to achieve it. In this paper, we propose simple metrics to measure fault tolerance of ANNs for classification tasks and address the issue of redundancy. Our approach also reveals a …